Large search&replace-all operations easily kill jEdit because of OutOfMemory errors.
Give the user the option to disable the undo/redo log for search&replace all operations.
A prototype implementation is attached in patch https://sourceforge.net/tracker/?func=detail&aid=3529803&group_id=588&atid=300588
A example file to test this is attached: search for ',' and replace all with ', '
(note the extra space).
The OOM error happens in the AWT thread and is not catched by the normal jEdit exception
handling, so the user never get the response of the OutOfMemory error. The java process
just continues to burn cpu and never finishes.
Submitted | thomasmey - 2012-05-26 - 09:36:18z | Assigned | nobody |
---|---|---|---|
Priority | 5 | Category | None |
Status | Open | Group | None |
Resolution | None | Visibility | No |
2012-05-26 - 11:12:55z jarekczek |
Thomas, could you desribe where you suggest to put the new option? So that we know that without compiling patched jedit. |
---|---|
2012-05-26 - 11:22:03z thomasmey |
The option to disable the undo/redo log for the search&replace-all operation should be a checkbox in the settings of the search dialog. When the checkbox is set the next search&replace-all operation is performed without an undo/redo log. When the checkbox is not set, the next search&replace-all operation should work as is. The checkbox should only influence the search&replace-all operation. |
2012-05-27 - 21:21:29z jarekczek |
This checkbox, useful in some situations, may still be controversial. I think I have
an idea of having the functionality in a way that no-one would object. Hypersearch after 1K hits displays a dialog asking whether to stop searching. It would be good to find how this limit is hardcoded and use it for our purposes. After 1000 replacements are done a dialog should appear like that: ===== Do you want to keep the undo history? The operation is treated as an undoable event by default, however clearing the undo information about this and previous activities would speed up the search. Keep undo / No undo ===== What's important, to make user interface most friendly, is to have 2 buttons labelled as I wrote, not Yes / No. If you are able to implement that, Thomas, there will be only technical details about the patch left. Interface would be harmless, so we can introduce it without a delay. Note that you must erase the whole undo information, not just skip the current one. Right after the s&r, performed without undo, no undo will be possible. |
2012-05-29 - 18:42:02z thomasmey |
I really like your idea and I'll implement something over this week. |
2012-05-30 - 05:36:50z thomasmey |
It seems as no mail get's send out, when you just attach a patch to a request. so: first prototype patch attached, that still has some glitches. |
2012-05-31 - 19:49:09z thomasmey |
Hello, here some numbers and results based on attached prototype patch: 10.000 lines (2.260.000 changes "," -> ", ") 21:10:30 [AWT-EventQueue-0] [debug] AWT-EventQueue-0: Call= 1 DiffTime=13.148.368.127 DiffUsedMem= 6.111.360 - Drop Undo 21:11:15 [AWT-EventQueue-0] [debug] AWT-EventQueue-0: Call= 2 DiffTime=8.980.303.341 DiffUsedMem= 372.544.648 - Keep Undo 21:12:19 [AWT-EventQueue-0] [debug] AWT-EventQueue-0: Call= 3 DiffTime=3.660.864.354 DiffUsedMem= 20.112.424 - Drop Undo 21:13:39 [AWT-EventQueue-0] [debug] AWT-EventQueue-0: Call= 4 DiffTime=4.553.024.598 DiffUsedMem= 360.235.976 - Keep Undo 50.000 lines (11.300.000 changes "," -> ", ") 21:17:33 [AWT-EventQueue-0] [debug] AWT-EventQueue-0: Call= 5 DiffTime=37.807.525.594 DiffUsedMem= 47.003.104 - Drop Undo 21:21:34 [AWT-EventQueue-0] [debug] AWT-EventQueue-0: Call= 6 DiffTime=31.706.368.273 DiffUsedMem= 46.073.128 - Drop Undo 10.000 lines (2.260.000 changes "," -> ", ") 21:22:53 [AWT-EventQueue-0] [debug] AWT-EventQueue-0: Call= 7 DiffTime=1.703.767.398 DiffUsedMem= 17.378.504 - Drop undo 50.000 lines (11.300.000 changes "," -> ", ") 21:26:38 [AWT-EventQueue-0] [debug] AWT-EventQueue-0: Call= 8 DiffTime=23.312.674.185 DiffUsedMem= 36.395.432 - Drop undo Keep undo never finishes for 50.000 lines: 21:37:11 [AWT-XAWT] [error] AWT-XAWT: Mai 31, 2012 9:37:11 PM sun.awt.X11.XToolkit processException 21:37:11 [AWT-XAWT] [error] AWT-XAWT: Warnung: Exception on Toolkit thread 21:37:11 [AWT-XAWT] [error] AWT-XAWT: java.lang.OutOfMemoryError: GC overhead limit exceeded 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at java.util.Arrays.copyOf(Arrays.java:2367) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at java.lang.StringBuilder.append(StringBuilder.java:132) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XEvent.getFieldsAsString(XEvent.java:117) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XWrapperBase.toString(XWrapperBase.java:37) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XEvent.toString(XEvent.java:8) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at java.lang.String.valueOf(String.java:2902) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at sun.util.logging.PlatformLogger$JavaLogger.doLog(PlatformLogger.java:524) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at sun.util.logging.PlatformLogger.finest(PlatformLogger.java:310) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XComponentPeer.isEventDisabled(XComponentPeer.java:1260) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XDecoratedPeer.isEventDisabled(XDecoratedPeer.java:942) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XContentWindow.isEventDisabled(XContentWindow.java:103) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XWindow.handleMotionNotify(XWindow.java:799) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XBaseWindow.dispatchEvent(XBaseWindow.java:1096) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XBaseWindow.dispatchToWindow(XBaseWindow.java:1066) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XToolkit.dispatchEvent(XToolkit.java:565) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XToolkit.run(XToolkit.java:674) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XToolkit.run(XToolkit.java:595) 21:37:11 [AWT-XAWT] [error] AWT-XAWT: at java.lang.Thread.run(Thread.java:722) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: Mai 31, 2012 9:37:27 PM sun.awt.X11.XToolkit processException 21:37:27 [AWT-XAWT] [error] AWT-XAWT: Warnung: Exception on Toolkit thread 21:37:27 [AWT-XAWT] [error] AWT-XAWT: java.lang.OutOfMemoryError: GC overhead limit exceeded 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at java.util.Arrays.copyOf(Arrays.java:2367) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at java.lang.StringBuilder.append(StringBuilder.java:132) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XEvent.getFieldsAsString(XEvent.java:106) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XWrapperBase.toString(XWrapperBase.java:37) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XEvent.toString(XEvent.java:8) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at java.lang.String.valueOf(String.java:2902) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at sun.util.logging.PlatformLogger$JavaLogger.doLog(PlatformLogger.java:524) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at sun.util.logging.PlatformLogger.finest(PlatformLogger.java:310) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XComponentPeer.isEventDisabled(XComponentPeer.java:1260) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XDecoratedPeer.isEventDisabled(XDecoratedPeer.java:942) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XContentWindow.isEventDisabled(XContentWindow.java:103) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XWindow.handleMotionNotify(XWindow.java:799) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XBaseWindow.dispatchEvent(XBaseWindow.java:1096) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XBaseWindow.dispatchToWindow(XBaseWindow.java:1066) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XToolkit.dispatchEvent(XToolkit.java:565) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XToolkit.run(XToolkit.java:674) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XToolkit.run(XToolkit.java:595) 21:37:27 [AWT-XAWT] [error] AWT-XAWT: at java.lang.Thread.run(Thread.java:722) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: Mai 31, 2012 9:37:52 PM sun.awt.X11.XToolkit processException 21:37:52 [AWT-XAWT] [error] AWT-XAWT: Warnung: Exception on Toolkit thread 21:37:52 [AWT-XAWT] [error] AWT-XAWT: java.lang.OutOfMemoryError: GC overhead limit exceeded 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at java.util.Arrays.copyOfRange(Arrays.java:2694) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at java.lang.String.<init>(String.java:234) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at java.lang.StringBuilder.toString(StringBuilder.java:405) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XEvent.getFieldsAsString(XEvent.java:116) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XWrapperBase.toString(XWrapperBase.java:37) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XEvent.toString(XEvent.java:8) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at java.lang.String.valueOf(String.java:2902) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at sun.util.logging.PlatformLogger$JavaLogger.doLog(PlatformLogger.java:524) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at sun.util.logging.PlatformLogger.finest(PlatformLogger.java:310) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XComponentPeer.isEventDisabled(XComponentPeer.java:1260) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XDecoratedPeer.isEventDisabled(XDecoratedPeer.java:942) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XContentWindow.isEventDisabled(XContentWindow.java:103) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XWindow.handleMotionNotify(XWindow.java:799) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XBaseWindow.dispatchEvent(XBaseWindow.java:1096) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XBaseWindow.dispatchToWindow(XBaseWindow.java:1066) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XToolkit.dispatchEvent(XToolkit.java:565) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XToolkit.run(XToolkit.java:674) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XToolkit.run(XToolkit.java:595) 21:37:52 [AWT-XAWT] [error] AWT-XAWT: at java.lang.Thread.run(Thread.java:722) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: Mai 31, 2012 9:38:08 PM sun.awt.X11.XToolkit processException 21:38:08 [AWT-XAWT] [error] AWT-XAWT: Warnung: Exception on Toolkit thread 21:38:08 [AWT-XAWT] [error] AWT-XAWT: java.lang.OutOfMemoryError: GC overhead limit exceeded 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at java.util.Arrays.copyOf(Arrays.java:2367) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at java.lang.StringBuilder.append(StringBuilder.java:132) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XMotionEvent.getFieldsAsString(XMotionEvent.java:86) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XWrapperBase.toString(XWrapperBase.java:37) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XMotionEvent.toString(XMotionEvent.java:8) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at java.lang.String.valueOf(String.java:2902) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at java.lang.StringBuilder.append(StringBuilder.java:128) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XEvent.getFieldsAsString(XEvent.java:90) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XWrapperBase.toString(XWrapperBase.java:37) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XEvent.toString(XEvent.java:8) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at java.lang.String.valueOf(String.java:2902) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at sun.util.logging.PlatformLogger$JavaLogger.doLog(PlatformLogger.java:524) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at sun.util.logging.PlatformLogger.finest(PlatformLogger.java:310) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XComponentPeer.isEventDisabled(XComponentPeer.java:1260) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XDecoratedPeer.isEventDisabled(XDecoratedPeer.java:942) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XContentWindow.isEventDisabled(XContentWindow.java:103) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XWindow.handleMotionNotify(XWindow.java:799) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XBaseWindow.dispatchEvent(XBaseWindow.java:1096) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XBaseWindow.dispatchToWindow(XBaseWindow.java:1066) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XToolkit.dispatchEvent(XToolkit.java:565) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XToolkit.run(XToolkit.java:674) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at sun.awt.X11.XToolkit.run(XToolkit.java:595) 21:38:08 [AWT-XAWT] [error] AWT-XAWT: at java.lang.Thread.run(Thread.java:722) 21:38:31 [AWT-XAWT] [error] AWT-XAWT: Mai 31, 2012 9:38:31 PM sun.awt.X11.XToolkit processException 21:38:31 [AWT-XAWT] [error] AWT-XAWT: Warnung: Exception on Toolkit thread 21:38:31 [AWT-XAWT] [error] AWT-XAWT: java.lang.OutOfMemoryError: GC overhead limit exceeded 21:38:48 [AWT-XAWT] [error] AWT-XAWT: Mai 31, 2012 9:38:48 PM sun.awt.X11.XToolkit processException 21:38:48 [AWT-XAWT] [error] AWT-XAWT: Warnung: Exception on Toolkit thread 21:38:48 [AWT-XAWT] [error] AWT-XAWT: java.lang.OutOfMemoryError: GC overhead limit exceeded 21:39:12 [AWT-XAWT] [error] AWT-XAWT: Mai 31, 2012 9:39:12 PM sun.awt.X11.XToolkit processException 21:39:12 [AWT-XAWT] [error] AWT-XAWT: Warnung: Exception on Toolkit thread 21:39:12 [AWT-XAWT] [error] AWT-XAWT: java.lang.OutOfMemoryError: GC overhead limit exceeded [ and so on ...] |
2012-05-26 - 09:36:19z thomasmey |
jedit-slow-search-replace.txt.zip example file to demonstrate the behaviour |
---|---|
2012-05-26 - 11:27:15z thomasmey |
jEdit-disable-undo-preview.png |
2012-05-29 - 21:17:24z thomasmey |
jEdit-disableundo-ask-user.patch First prototype with some glitches |